Papers with language comprehension
Understanding language-elicited EEG data by predicting it from a fine-tuned language model (N19-1)
Copied to clipboard
| Challenge: | Existing studies have only found two of the ERPs to be predictable from embeddings of a stream of language. |
| Approach: | They propose to fine tune a language model to predict ERPs by embedding a stream of language into a model that allows them to be more accurate. |
| Outcome: | The proposed model fine tunes the ERPs to predict them for the first time. |
Revisiting the Uniform Information Density Hypothesis (2021.emnlp-main)
Copied to clipboard
| Challenge: | The uniform information density hypothesis posits a preference among language users for utterances structured such that information is distributed uniformly across a signal. |
| Approach: | They propose to test the hypothesis by using reading time and acceptability data to examine the effect of surprisal on language comprehension and acceptabilities. |
| Outcome: | The proposed hypothesis makes predictions about language comprehension and linguistic acceptability . |
AC-EVAL: Evaluating Ancient Chinese Language Understanding in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | AC-EVAL is a benchmark designed to assess the advanced knowledge and reasoning capabilities of LLMs within the context of ancient Chinese. |
| Approach: | They propose a benchmark to assess the advanced knowledge and reasoning capabilities of LLMs in ancient Chinese. |
| Outcome: | AC-EVAL aims to assess the comprehension of ancient Chinese texts . the benchmark covers 13 tasks covering historical facts, geography, social customs, art, philosophy, classical poetry and prose. |
LLM-Personalize: Aligning LLM Planners with Human Preferences via Reinforced Self-Training for Housekeeping Robots (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models have shown significant potential for robotics tasks, but a gap remains in personalization of LLMs to household preferences. |
| Approach: | They propose a framework to personalize LLM planners for household robotics . they use imitation learning and reinforced self-training to personalise the planner . |
| Outcome: | The proposed framework performs iterative planning in multi-room, partially-observable household environments, utilizing a scene graph built dynamically from local observations. |
Modelling Language Acquisition through Syntactico-Semantic Pattern Finding (2023.findings-eacl)
Copied to clipboard
| Challenge: | Usage-based theories of language acquisition have documented the processes by which children acquire language through communicative interaction. |
| Approach: | They propose a method for learning grammars based on similarities and differences in linguistic observations alone. |
| Outcome: | The proposed method is able to learn compositional lexical and item-based constructions of variable extent and degree of abstraction, along with a network of emergent syntactic categories. |
Improving Multilingual Instruction Finetuning via Linguistically Natural and Diverse Datasets (2024.findings-emnlp)
Copied to clipboard
Sathish Reddy Indurthi, Wenxuan Zhou, Shamil Chollampatt, Ravi Agrawal, Kaiqiang Song, Lingxiao Zhao, Chenguang Zhu
| Challenge: | Advancements in Large Language Models (LLMs) have significantly enhanced instruction-following capabilities, but most IFT datasets are predominantly in English, limiting model performance in other languages. |
| Approach: | They propose a method for collecting multilingual IFT datasets that preserves linguistic naturalness and ensures prompt diversity. |
| Outcome: | Experiments show that LLMs fine-tuned using this method show significant improvements in generative and discriminative tasks. |
Coreference information guides human expectations during natural reading (2020.coling-main)
Copied to clipboard
| Challenge: | Existing theories of coreference processing focus on memory, but some theories focus on expectations. |
| Approach: | They hypothesize that coreference tracking also informs human expectations about upcoming words. |
| Outcome: | The proposed coreference-aware parser improves human response times in a naturalistic reading experiment. |
John praised Mary because _he_? Implicit Causality Bias and Its Interaction with Explicit Cues in LMs (2021.findings-acl)
Copied to clipboard
| Challenge: | Psycholinguists have identified one such cue in the implicit causality bias of interpersonal verbs. |
| Approach: | They propose to use pre-trained language models to encode IC bias at inference time . they hypothesize that when a cause is explicitly stated, an incongruent IC biased leads to a delay in human processing. |
| Outcome: | The results suggest that pre-trained language models tend to prioritize lexical patterns over higher-order signals. |
Meaning Beyond Truth Conditions: Evaluating Discourse Level Understanding via Anaphora Accessibility (2025.acl-long)
Copied to clipboard
| Challenge: | Existing assessments of understanding at the lexical and sentence levels are limited to lexica and sentence level, but few of them target whether LLMs accurately represent and update states of natural language discourse. |
| Approach: | They propose anaphora accessibility as a diagnostic for assessing discourse understanding . they use a dataset inspired by theoretical research in dynamic semantics to evaluate human and LLM performance. |
| Outcome: | The proposed dataset shows that humans and LLMs align on some tasks and diverge on others. |
Towards Verifiable Text Generation with Evolving Memory and Self-Reflection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) often produce factually incorrect information, also known as hallucination. |
| Approach: | They propose a framework for verifiable text generation with evolving memory and self-reflection that incorporates long-term memory to retain documents and recent documents. |
| Outcome: | The proposed framework outperforms baselines on five datasets across three knowledge-intensive tasks. |
Is the Brain Mechanism for Hierarchical Structure Building Universal Across Languages? An fMRI Study of Chinese and English (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that the brain builds hierarchical syntactic structures, but it is unknown whether they are universal across languages. |
| Approach: | They analyze the working memory requirements when applying parsing strategies to two languages: Chinese and English. |
| Outcome: | The proposed method shows that the brain adopts parsing strategies with less memory load according to different language structures. |
Rule-KBQA: Rule-Guided Reasoning for Complex Knowledge Base Question Answering with Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for knowledge base question answering lack grammaticality, faithfulness, and controllability due to hallucinations in the reasoning process. |
| Approach: | They propose a framework that employs learned rules to guide the generation of logical forms. |
| Outcome: | The proposed method achieves competitive results on standard KBQA datasets. |
Text Genre and Training Data Size in Human-like Parsing (D19-1)
Copied to clipboard
| Challenge: | Using domain-specific training, NLP systems work better, but only when the training examples come from the same textual genre. |
| Approach: | They relate the states of a neural phrase-structure parser to electrophysiological measures from human participants. |
| Outcome: | The proposed model is well-matched to the training data from human participants, but only when the training examples come from the same genre. |
Disentangling Text and Math in Word Problems: Evidence for the Bidimensional Structure of Large Language Models’ Reasoning (2025.findings-acl)
Copied to clipboard
Pedro Calais, Gabriel Franco, Zilu Tang, Themistoklis Nikas, Wagner Meira Jr., Evimaria Terzi, Mark Crovella
| Challenge: | Existing studies show that LLMs struggle with text interpretation and equation solving, despite distinct proficiencies in textual and mathematical components. |
| Approach: | They disentangle textual interpretation and mathematical solving steps in word problems drawn from Brazil's largest college entrance exam and popular grade school-level benchmark GSM8K. |
| Outcome: | The proposed model outperforms LLMs in Brazil's largest college entrance exam and popular grade school-level benchmark. |
Small But Funny: A Feedback-Driven Approach to Humor Distillation (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been used to transfer knowledge from LLMs to smaller, smaller language models (SLMs). |
| Approach: | They propose to assign a dual role to the LLM as a “teacher” generating data, as well as evaluating the student’s performance. |
| Outcome: | The proposed approach narrows the performance gap between LLMs and larger models by incorporating feedback into the data. |
CoGen: Learning from Feedback with Coupled Comprehension and Generation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on coupling comprehension and generation in computational systems show that the ability to finish incomplete partner utterances in dialogue is closely related to comprehension and vice versa. |
| Approach: | They propose techniques to tightly integrate comprehension and generation capabilities with focus on continually learning from interaction with users. |
| Outcome: | The proposed models improve performance by 26% and 17% over time, while the non-coupled system is more human-like. |
1+1>2: A Synergistic Sparse and Low-Rank Compression Method for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Low-rank approximation compresses the model by retaining its essential structure with minimal information loss. |
| Approach: | They propose a method that leverages the strengths of pruning and low-rank approximation for LLMs. |
| Outcome: | The proposed methods surpass the existing methods on LLaMA and Qwen2.5 models. |
InteRead: An Eye Tracking Dataset of Interrupted Reading (2024.lrec-main)
Copied to clipboard
| Challenge: | Eye movements during reading can provide insights into cognitive processes and language comprehension, but the scarcity of reading data with interruptions hampers advances in the development of intelligent learning technologies. |
| Approach: | They propose a dataset of eye movements during reading that includes eye movements and word frequency effects. |
| Outcome: | The proposed dataset shows that interruptions, word length and word frequency effects significantly impact eye movements during reading. |
On the Same Wavelength? Evaluating Pragmatic Reasoning in Language Models across Broad Concepts (2025.emnlp-main)
Copied to clipboard
| Challenge: | Language models (LMs) are increasingly used as conversational agents because of their pragmatic reasoning abilities. |
| Approach: | They propose an evaluation framework derived from *Wavelength*, a popular communication game where a speaker and a listener communicate about a broad range of concepts in a granular manner. |
| Outcome: | The proposed evaluation framework outperforms direct and Chain-of-Thought (CoT) prompting on language comprehension and language production tasks. |
Multimodal Language Models Show Evidence of Embodied Simulation (2024.lrec-main)
Copied to clipboard
| Challenge: | Multimodal large language models (MLLMs) are gaining popularity as partial solutions to the “symbol grounding problem” faced by language models trained on text alone. |
| Approach: | They propose to use multimodal large language models to integrate linguistic representations with data from other modalities to investigate whether they are integrated into a model. |
| Outcome: | The proposed models are sensitive to visual features like object shape when it is implied by a verbal description of an event. |
Clozing the Gap: Exploring Why Language Model Surprisal Outperforms Cloze Surprisal (2026.acl-long)
Copied to clipboard
| Challenge: | cloze task is being replaced with LM probabilities for how predictable a word is in its context . clozen task is used to measure how predictable word is compared to unexpected words . |
| Approach: | They propose to use LM probabilities to quantify how predictable a word is . they propose to test whether LMs are better at distinguishing semantically similar words . |
| Outcome: | The results show that LM probabilities outperform cloze probabilities for the right reasons . they also show that human-like prediction is more sensitive to LM probability distinctions . |
AraEval: An Arabic Multi-Task Evaluation Suite for Large Language Models (2025.emnlp-main)
Copied to clipboard
Alhanoof Althnian, Norah A. Alzahrani, Shaykhah Z. Alsubaie, Eman Albilali, Ahmed Abdelali, Nouf M. Alotaibi, M Saiful Bari, Yazeed Alnumay, Abdulhamed Alothaimen, Maryam Saif, Shahad D. Alzaidi, Faisal Abdulrahman Mirza, Yousef Almushayqih, Mohammed Al Saleem, Ghadah Alabduljabbar, Abdulmohsen Al-Thubaity, Areeb Alowisheq, Nora Al-Twairesh
| Challenge: | AraEval is a suite of evaluation tasks designed to assess the advanced knowledge, reasoning, truthfulness, and instruction following capabilities of large language models. |
| Approach: | They propose to use AraEval to assess the advanced knowledge, reasoning, truthfulness, and instruction following capabilities of large language models in the Arabic context. |
| Outcome: | The evaluation suite covers a broad spectrum of domains, including science, history, religion, and literature. |